Approximating the model of a water distribution network as a Markov decision process

In this paper, our objective is to convert the model of a water distribution network described via Stochastic differential equations (SDE) into Markov decision process (MDP). The motivation behind work that while MDP's represents underlying dynamics for dynamic programming and reinforcement learning, actual best equations, therefore, we would like SDE MDP. We have applied Kushner's chain approximation (MCA) method verified it using novel modified Monte Carlo which can be considered as an alternative well-known MCA. Both methods approximate value function simulation studies show obtained functions from both converge almost same value.